How to Use a Local AI Model on an On-premises Private Server

You can configure a Local LLM model and a Local Embedding model separately on an On-premises Private Server so that AI processing runs within your local infrastructure without connecting to external services. This improves data security and supports offline environments.

Configuring the Local LLM Model

Configure the following parameters in ragic.properties:

1. LOCAL_AI_URL=

Required. Enter the API endpoint of your Local AI model service. It must be compatible with the OpenAI API format, for example: http://localhost:11434/v1/.

2. LOCAL_AI_KEY=

Optional. Enter the API key if required by your AI service. Most Local AI model services do not require this setting.

Note: Once LOCAL_AI_URL is configured, the system will always use the Local AI model and will not switch to cloud AI services.

Supported Model Prefixes

The system automatically identifies and maps models based on their name prefixes and supports the following common models:

Configuring the Local Embedding Model

You can also configure a Local Embedding model so that semantic search runs on your local infrastructure. If not configured, semantic search will use the cloud Embedding model instead.

Configure the following parameters in ragic.properties:

1. LOCAL_EMBEDDING_URL=

Required. Enter the API endpoint of your Local Embedding model service, for example: http://localhost:11434/api/embeddings.

2. LOCAL_EMBEDDING_MODEL=

Required. Enter the Embedding model name, for example: qwen3-embedding:0.6b.